Summary
Rate limiting rejects or delays requests above a defined quota. Throttling slows clients down to protect capacity or enforce fairness.
Interview Points
- Rate limiting is quota enforcement: requests per user, IP, token, or tenant.
- Throttling is dynamic slowing, often based on load or policy.
- Common algorithms: token bucket, leaky bucket, fixed window, sliding window.
- Return clear signals like HTTP 429 and
Retry-After. - Apply limits at API gateway, service layer, or per-resource boundary.
2-3 Minute Interview Script
“I separate rate limiting and throttling by intent. Rate limiting enforces a quota, like 100 requests per minute per API key. Throttling slows or delays traffic, often when a system or tenant is consuming too much capacity.
In a design, I would choose the limit dimension carefully: user, IP, token, tenant, endpoint, or resource. I would usually implement coarse limits at the API gateway and finer limits near the service that owns the resource.
Token bucket is a common choice because it allows bursts while preserving an average rate. For clients, the system should return 429 with a retry hint so well-behaved callers can back off.
The senior point is fairness. Limits protect the system, but they also protect other users from noisy neighbors.”
Follow-Ups
- Token bucket vs leaky bucket?
- Where should rate limiting live?